Skip to content

gh-61206: Support zstandard compression in the zipimport module #138254

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Sep 2, 2025
Merged

gh-61206: Support zstandard compression in the zipimport module #138254

merged 3 commits into from
Sep 2, 2025
+112 −25

Conversation

gpshead
Copy link
Member

@gpshead gpshead commented Aug 30, 2025 •
edited
Loading

There's long been a request to support the alternate compression methods in zipimport. zstandard makes sense as it can either be fast or high compression.

While we "could"... It is probably not worth the effort to do the same for bz2 or lzma zip entries in zipimport's implementation at this point.

📚 Documentation preview 📚: https://cpython-previews--138254.org.readthedocs.build/

@bedevere-app bedevere-app bot mentioned this pull request Aug 30, 2025
Closed
emmatyping
emmatyping approved these changes Sep 1, 2025
View reviewed changes
Copy link
Member

@emmatyping emmatyping left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exciting! This would be really useful for compatibility if wheels can adopt using zstd in the future.

@gpshead gpshead merged commit a2ba0a7 into python:main Sep 2, 2025
88 of 90 checks passed
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 CentOS9 NoGIL Refleaks 3.x (tier-1) has failed when building commit a2ba0a7.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1610/builds/1997) and take a look at the build logs.
  4. Check if the failure is related to this commit (a2ba0a7) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1610/builds/1997

Failed tests:

  • test_free_threading

Summary of the results of the build (if available):

==

Click to see traceback logs 
remote: Enumerating objects: 14, done.        
remote: Counting objects:  10% (1/10)        
remote: Counting objects:  20% (2/10)        
remote: Counting objects:  30% (3/10)        
remote: Counting objects:  40% (4/10)        
remote: Counting objects:  50% (5/10)        
remote: Counting objects:  60% (6/10)        
remote: Counting objects:  70% (7/10)        
remote: Counting objects:  80% (8/10)        
remote: Counting objects:  90% (9/10)        
remote: Counting objects: 100% (10/10)        
remote: Counting objects: 100% (10/10), done.        
remote: Compressing objects:  10% (1/10)        
remote: Compressing objects:  20% (2/10)        
remote: Compressing objects:  30% (3/10)        
remote: Compressing objects:  40% (4/10)        
remote: Compressing objects:  50% (5/10)        
remote: Compressing objects:  60% (6/10)        
remote: Compressing objects:  70% (7/10)        
remote: Compressing objects:  80% (8/10)        
remote: Compressing objects:  90% (9/10)        
remote: Compressing objects: 100% (10/10)        
remote: Compressing objects: 100% (10/10), done.        
remote: Total 14 (delta 0), reused 0 (delta 0), pack-reused 4 (from 2)        
From https://github.com/python/cpython
 * branch                    main       -> FETCH_HEAD
Note: switching to 'a2ba0a7552580f616f74091f8976410f8a310313'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at a2ba0a75525 gh-61206: Support zstandard compression in the zipimport module (GH-138254)
Switched to and reset branch 'main'

configure: WARNING: no system libmpdec found; falling back to pure-Python version for the decimal module

make: *** [Makefile:2486: buildbottest] Error 2

picnixz
picnixz reviewed Sep 2, 2025
View reviewed changes
Lib/zipimport.py
_importing_zstd = True
try:
from _zstd import ZstdDecompressor as _zstd_decompressor_class
except Exception:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the rationale of catching all exceptions here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

matching the pattern of what the existing code did for zlib. realistically likely only an ImportError.

@brettcannon brettcannon removed their request for review September 3, 2025 19:11
lkollar pushed a commit to lkollar/cpython that referenced this pull request Sep 9, 2025
@gpshead @lkollar
pythongh-61206: Support zstandard compression in the zipimport module (…
208de43 
…pythonGH-138254)

* pythongh-61206: support zstd in zipimport

* NEWS entry

* versionchanged doc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@picnixz picnixz picnixz left review comments

@emmatyping emmatyping emmatyping approved these changes

@ericsnowcurrently ericsnowcurrently Awaiting requested review from ericsnowcurrently ericsnowcurrently is a code owner

@ncoghlan ncoghlan Awaiting requested review from ncoghlan ncoghlan is a code owner

@warsaw warsaw Awaiting requested review from warsaw warsaw is a code owner

Assignees

@emmatyping emmatyping

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@gpshead @bedevere-bot @emmatyping @picnixz